Evaluation of Information and Knowledge Retrieval Systems
Maryam Karami; ٖGolnessa Galyani-Moghaddam; Farhad Hadinejad
Abstract
IntroductionThis study evaluates and ranks the scientific journals of psychology using the Data Envelopment Analysis (DEA) model. Data envelopment analysis (DEA) is a linear programming-based technique for measuring the relative performance of organizational units where the presence of multiple inputs ...
Read More
IntroductionThis study evaluates and ranks the scientific journals of psychology using the Data Envelopment Analysis (DEA) model. Data envelopment analysis (DEA) is a linear programming-based technique for measuring the relative performance of organizational units where the presence of multiple inputs and outputs makes comparison difficult. DEA was originally developed to measure the efficiency of a Decision Making Units (DMU) as a whole unit, without considering its internal structure. In other words, the system is treated as a black box, within which inputs are supplied to produce outputs, with there generally being a positive correlation between the two.Traditional DEA models are only able to evaluate DMUs with deterministic inputs and outputs, while in real-world problems, data are usually uncertain. So far, various approaches have been introduced to overcome the uncertainty of data. In this paper, two robust DEA models are presented to evaluate the performance of systems with continuous uncertain data under constant return to scale (CRS) and variable return to scale (VRS) conditions. The main advantage of the proposed robust DEA models over the previous robust DEA models is that they are able to formulate uncertainty in both input and output data. Moreover, these models are also developed directly on basic traditional DEA models (not alternative models).Research Question(s)What is the status of effective indicators (citation and non-citation) in the ranking of valid scientific journals of psychology?What are the top scientific journals in different fields of psychology?What is the ranking status of psychology journals using the Data Envelopment Analysis (DEA) model?What is the ranking status of psychology scientific journals based on the Anderson-Peterson CRS and Anderson-Peterson VRS models?Literature ReviewDEA is used for ranking journals in different fields. A study by Rosenthal and Weiss (2017) ranks business journals, using data from the Thomson Reuters Journal Citation Reports® (JCR). They ranked 358 business journals from five different JCR categories according to such citation-based factors as the number of articles, the number of citations, impact factor, five-year impact factor, immediacy index, eigenfactor score, and article influence score. They compute the relative efficiencies of the journals and thereby create plausible journal rankings that largely, but not completely, corroborate three widely used business publication journal ranking lists. In addition, we show how the different characteristics of the JCR data impact the DEA ranking model. Finally, they identify journals that are not on the business publication lists but consistently perform very well relative to those benchmark journals, and should possibly be included in the business publication ranking lists. They also identify journals whose inclusion in widely used business publication rankings cannot be justified by their methodology.MethodologyIn this study of data envelopment analysis technique, Anderson-Peterson model, CRS and VRS output-oriented, 2018 JCR ratings scientific journals of psychology were evaluated and the top journals ranked. This study has a quantitative approach and is applied research. The research method is scientometrics and bibliometrics using Operational Research techniques. In order to evaluate and rank psychology scientific journals, seven citation indicators were extracted from the Analytics Clarivate database. These indicators are the total number of citations, journal impact factor, five-year impact factor, immediate index, citeable items, journal influence factor, and journal article influence factor.In this study, using data envelopment analysis technique, Anderson-Peterson model, CRS and VRS output-oriented, 2018 JCR ratings scientific journals of psychology were evaluated and the top journals ranked.ResultsFindings show that with the help of JCR indicators and using the Anderson-Peterson CRS and VRS model, more accurate rankings of journals can be achieved. The comparison of scientific journals is in the ranking of CRS and VRS models. The Journal of Psychological Science in the Public Interest tops in both models. The top 10 journals based on CRS respectively are: Psychological Science in the Public Interest, International Review of Sport and Exercise Psychology, Annual Review of Psychology, Psychological Bulletin, Psychological Inquiry, Behavioral and Brain Sciences, Social Issues and Policy Review, Personality and Social Psychology Review, Annual Review of Clinical Psychology, Journal of Personality and Social Psychology.The top 10 journals based on VRS respectively are: Psychological Science in the Public Interest, International Review of Sport and Exercise Psychology, Annual Review of Psychology, Frontiers in Psychology, Psychological Bulletin, Journal of Personality and Social Psychology, Trends in Cognitive Sciences, Psychological Science, Frontiers in Human Neurosciences.DiscussionHere goes the text for your discussion.ConclusionScientific journals of psychology were evaluated and ranked using the Anderson-Peterson CRS and VRS models. Also, the evaluation indicators of scientific journals in the Clarivate Analytics database were reviewed and the top three journals in 11 areas of psychology and the top 30 journals in this field were identified and introduced. The top 30 journals and their differences in CRS and VRS models were compared and analyzed. In the part of descriptive findings, the frequency of journals in different fields of psychology was found, including 10.25% of psychology journals in general, 10.91% of applied psychology journals, 1.86% of biological psychology journals, 17.17% scientific journals in the clinical psychology field, 10.11% scientific journals in advanced psychology field, 7.85% scientific journals in educational psychology field, 11.71% scientific journals in experimental psychology field, 1.73% scientific journals in mathematics psychology field, 18.28% are scientific journals in the field of interdisciplinary psychology, 1.73% are scientific journals in the field of psychoanalytic psychology, and 8.38% are scientific journals in the field of social psychology. Finally, considering the needs of the country's scientific community, conducting research with other research communities (different scientific fields and disciplines), conducting research and ranking journals with other indicators and normalized indicators are recommended.
